Papers with rank reduction
Beyond Variance: Knowledge-Aware LLM Compression via Fisher-Aligned Subspace Diagnostics (2026.acl-long)
Copied to clipboard
| Challenge: | Existing methods for activation compression are gradient-blind and preserve high-variance dimensions regardless of their impact on factual knowledge preservation. |
| Approach: | They propose a knowledge-aware compression framework that models activation-gradient coupling by directly modeling subspaces. |
| Outcome: | The proposed framework preserves 6–8% more accuracy on knowledge-intensive benchmarks compared to variance-based methods at 50% rank reduction. |